Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(write path): newtype to enforce use of fully initialized slices #8717

Merged
merged 6 commits into from
Aug 14, 2024

Conversation

problame
Copy link
Contributor

@problame problame commented Aug 13, 2024

The tokio_epoll_uring::Slice / tokio_uring::Slice type is weird.
The new FullSlice newtype is better. See the doc comment for details.

The naming is not ideal, but we'll clean that up in a future refactoring where we move the FullSlice into tokio_epoll_uring. Then, we'll do the following:

  • tokio_epoll_uring::Slice is removed
  • FullSlice becomes tokio_epoll_uring::IoBufView
  • new type tokio_epoll_uring::IoBufMutView for the current tokio_epoll_uring::Slice<IoBufMut>

Context

I did this work in preparation for #8537.
There, I'm changing the type that the inmemory_layer.rs passes to DeltaLayerWriter::put_value_bytes and thus it seemed like a good opportunity to make this cleanup first.

@github-actions github-actions bot added the external A PR or Issue is created by an external user label Aug 13, 2024
@problame problame removed the external A PR or Issue is created by an external user label Aug 13, 2024
Copy link

github-actions bot commented Aug 13, 2024

2182 tests run: 2113 passed, 0 failed, 69 skipped (full report)


Flaky tests (1)

Postgres 16

  • test_delete_timeline_client_hangup: debug

Code coverage* (full report)

  • functions: 32.4% (7218 of 22256 functions)
  • lines: 50.4% (58291 of 115647 lines)

* collected from Rust tests only


The comment gets automatically updated with the latest test results
b55a70a at 2024-08-14T17:35:51.564Z :recycle:

@problame problame marked this pull request as ready for review August 13, 2024 20:02
@problame problame requested a review from a team as a code owner August 13, 2024 20:02
Copy link
Contributor

@VladLazar VladLazar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code looks good, but I have questions. I'd especially like to verify my understanding of the assertion

pageserver/src/tenant/blob_io.rs Outdated Show resolved Hide resolved
pageserver/src/tenant/ephemeral_file/page_caching.rs Outdated Show resolved Hide resolved
pageserver/src/tenant/storage_layer/inmemory_layer.rs Outdated Show resolved Hide resolved
pageserver/src/virtual_file.rs Show resolved Hide resolved
pageserver/src/virtual_file.rs Outdated Show resolved Hide resolved
pageserver/src/virtual_file/owned_buffers_io/io_buf_ext.rs Outdated Show resolved Hide resolved
@problame problame changed the title refactor(write path): take tokio_epoll_uring::Slice and require bytes_init() == bytes_total() refactor(write path): newtype to enforce use of fully initialized slices Aug 14, 2024
Copy link
Contributor

@VladLazar VladLazar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good!

@problame problame merged commit 168913b into main Aug 14, 2024
64 checks passed
@problame problame deleted the problame/refactor-write-path-take-slice branch August 14, 2024 19:57
bayandin added a commit that referenced this pull request Aug 15, 2024
## Problem

On macOS, clippy fails with the following error:

```
error: unused import: `crate::virtual_file::owned_buffers_io::io_buf_ext::IoBufExt`
  --> pageserver/src/tenant/remote_timeline_client/download.rs:26:5
   |
26 | use crate::virtual_file::owned_buffers_io::io_buf_ext::IoBufExt;
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |
   = note: `-D unused-imports` implied by `-D warnings`
   = help: to override `-D warnings` add `#[allow(unused_imports)]`
```

Introduced in #8717

## Summary of changes
- allow `unused_imports` for
`crate::virtual_file::owned_buffers_io::io_buf_ext::IoBufExt` on macOS
in download.rs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants